NEPy LIBRARY DEMO: $batch$

Neuroelectrics Corporation
Created November 2018
Modified: -

1. Description

`batch` is a funciton which you input a directory and it performs a Frida test for each of the files available. Such files have to be in any of these formats: `.easy`, `.easy.gz`, `.nedf` with a corresponding `.info` file if available. For each of the files, `batch` performs the following: ``` f = Frida(filepath, author=author, parameters=parameters) f.plotEEG() f.plotPSD() f.QC() f.preprocess(pipeline) f.QC() f.plotEEG() f.plotPSD() ```

2. Set up

When runnign the next cell, three input boxes are going to appear asking for the necessary data to set up:

Your name or ID. Example: MyName
The path where you have stored the NEpy repository. Copy paste the path into that box. Example: C:\Users\MyUser\Documents\Git\NEpy_v0.1.
The path where you have stored the file(s) you want to read (.easy, .easy.gz, .info). Example: C:\Users\MyUser\Documents\Git\NEpy_v0.1\sampledata

author = input('\033[1mUser name:\033[0m \n')
nepypath = input('\033[1mNEpy repository path:\033[0m \n ')
datapath = input('\033[1mData path:\033[0m \n ')

import sys
import os
import time
sys.path.append(nepypath)
from nepy.frida.frida import Frida
from nepy.frida.batch import processDirectory

print('\n')
%load_ext autoreload
%autoreload 2
print("______________________________________________")
print ("\nNotebook started:", time.strftime("%Y-%m-%d %H:%M"))

User name: 
Roser
NEpy repository path: 
 C:\Users\roser.sanchez\Documents\Git\NEpy_v0.1
Data path: 
 C:\Users\roser.sanchez\Desktop\EEG_DATA\testfiles


______________________________________________

Notebook started: 2019-04-15 16:01

3. Get help

Don't know how to deal with batch? No worries, ask for help!

The help function (similar to MATLAB) will print a description and an exaple of use of the batch function.

help(processDirectory)

Help on function processDirectory in module nepy.frida.batch:

processDirectory(datapath, author='anonymous', pipeline=None, parameters=None, plotit=True)
    Process all .easy or .easy.gz files in data's directory using Frida.
    :param datapath: directory of the folder containing the data.
    :param author: ('anonymous') user.
    :param pipeline: (['referenceData', 'detrendData', 'notch', 'filterDataA2B'])
    :param parameters: check Frida docstring for more information.
    :param plotit: flag to plot or not the data.
    :return: list of processed and skipped files.
    
    Example of use:
    >>> [processed, skipped] = processDirectory(datapath)

4. Preprocess a directory

It is really simple! You just need to copy the following line:

[processed, skipped] = processDirectory(datapath)

The arguments pipeline and parameters (see help(Frida)) are optional.

parameters = {
    'signal_offset_limit': 1.,
    'signal_std_limit': 15.,   
    'epoch_length': 10.,
    'epoch_amp_threshold': 75.,
    'epoch_std_threshold': 30.,
    'detrend_time': 10.,
    'line_freq': 50.,
    'Q_notch': 30.,
    'low_cutoff_freq': 2.,
    'high_cutoff_freq': 45.,
    'order': 5,
    'reference_electrodes': ['Cz']
}

[processed, skipped] = processDirectory(datapath, parameters=parameters)

Running with these arguments: {'plotit': True, 'parameters': {'signal_offset_limit': 1.0, 'signal_std_limit': 15.0, 'epoch_length': 10.0, 'epoch_amp_threshold': 75.0, 'epoch_std_threshold': 30.0, 'detrend_time': 10.0, 'line_freq': 50.0, 'Q_notch': 30.0, 'low_cutoff_freq': 2.0, 'high_cutoff_freq': 45.0, 'order': 5, 'reference_electrodes': ['Cz']}, 'pipeline': None, 'author': 'anonymous', 'datapath': 'C:\\Users\\roser.sanchez\\Desktop\\EEG_DATA\\testfiles'}


##########################################################################

         Processing C:\Users\roser.sanchez\Desktop\EEG_DATA\testfiles/20181214090159_testboard_E8.easy
##########################################################################


Found the file: C:\Users\roser.sanchez\Desktop\EEG_DATA\testfiles/20181214090159_testboard_E8.easy
Initializing in file path:   C:\Users\roser.sanchez\Desktop\EEG_DATA\testfiles/20181214090159_testboard_E8.easy
Processing:  20181214090159_testboard_E8
Filenameroot:  C:\Users\roser.sanchez\Desktop\EEG_DATA\testfiles/20181214090159_testboard_E8
Extension:  easy
 
_______________________________________________________________
Reading info file ...
Step Details
Info Version: 1.2
Step name: 
StartDate (firstEEGtimestamp): 1544774518507
Device class: ENOBIO
Communication type: Bluetooth
Device ID: 00:07:80:0F:63:86
Software's version: NIC v2.0.10
Firmware's version: 1256
Operative system: Windows
SDCard Filename: NONE
Additional channel: NONE

EEG Settings
Total number of channels: 8
Number of EEG channels: 8
Number of records of EEG: 90000
EEG sampling rate: 500 Samples/second
Effective EEG sampling rate: N/A
EEG recording configured duration (s): 180
Number of packets lost: 0(0.00%)
Line filter status: OFF
FIR filter status: OFF
EOG correction filter status: OFF
Reference filter status: OFF
EEG units: nV
EEG montage: 
	Channel 1: F6
	Channel 2: F4
	Channel 3: F2
	Channel 4: Fz
	Channel 5: F1
	Channel 6: F3
	Channel 7: F5
	Channel 8: Cz
Accelerometer data: ON
.info file found and read.

_______________________________________________________________
Reading: 20181214090159_testboard_E8 . easy 
 
Number of channels detected: 8
First sample recorded : 2018-12-14 09:01:58 

 L0 raw data data in uV
                 F6            F4            F2            Fz           F1  \
count  90000.000000  90000.000000  90000.000000  90000.000000  90000.00000   
mean     315.289488   -599.396540    223.635793   -282.218188   -268.81353   
std       15.718408     13.982947     13.640501     13.556471     13.62517   
min     -226.116000  -1118.230000   -289.630000   -789.690000   -776.76700   
25%      309.467000   -602.436000    221.395000   -284.433000   -271.36800   
50%      315.380000   -599.384000    223.636000   -282.239000   -268.79300   
75%      321.102000   -596.332000    225.830000   -280.046000   -266.31300   
max      888.443000    -80.871000    742.197000    231.695000    243.18600   

                 F3            F5            Cz            ax            ay  \
count  90000.000000  90000.000000  90000.000000  90000.000000  90000.000000   
mean     -32.343784   -515.676789    190.769976     81.849800     17.749811   
std       13.695445     13.813497     13.878209   1004.576048    821.329224   
min     -538.873000  -1024.389000   -322.151000  -4854.000000  -6423.000000   
25%      -35.047000   -518.655000    187.635000    -29.000000    186.000000   
50%      -32.329000   -515.651000    190.782000      0.000000    186.000000   
75%      -29.659000   -512.695000    193.882000      0.000000    225.000000   
max      479.459000     -3.576000    706.386000  10051.000000   4050.000000   

                 az       markers     unix_time  
count  90000.000000  90000.000000  9.000000e+04  
mean    9117.544778      0.000256  1.544775e+12  
std     1491.102339      0.016665  5.196253e+04  
min   -12081.000000      0.000000  1.544775e+12  
25%     9286.000000      0.000000  1.544775e+12  
50%     9326.000000      0.000000  1.544775e+12  
75%     9365.000000      0.000000  1.544775e+12  
max    11581.000000      2.000000  1.544775e+12  
Plotting EEG channels after this pipeline:
1 . Object created: 2019-04-15 16:02

File:  C:\Users\roser.sanchez\Desktop\EEG_DATA\testfiles/20181214090159_testboard_E8.easy

Plotting PSDs after this pipeline:
1 . Object created: 2019-04-15 16:02

File:  C:\Users\roser.sanchez\Desktop\EEG_DATA\testfiles/20181214090159_testboard_E8

Channel 1: F6, STD=  15.7 uV

Channel 2: F4, STD=  14.0 uV

Channel 3: F2, STD=  13.6 uV

Channel 4: Fz, STD=  13.6 uV

Channel 5: F1, STD=  13.6 uV

Channel 6: F3, STD=  13.7 uV

Channel 7: F5, STD=  13.8 uV

Channel 8: Cz, STD=  13.9 uV

--------QUALITY CHECK--------
Current log is:
['Object created: 2019-04-15 16:02']

Checking Quality of data with epoch length of  10.0  seconds.
Offset limit:  1.0
STD limit:  15.0
Channel F6    : Offset =    0.3 mV    / STD =   15.7 uV (*)
Channel F4    : Offset =   -0.6 mV    / STD =   14.0 uV  
Channel F2    : Offset =    0.2 mV    / STD =   13.6 uV  
Channel Fz    : Offset =   -0.3 mV    / STD =   13.6 uV  
Channel F1    : Offset =   -0.3 mV    / STD =   13.6 uV  
Channel F3    : Offset =   -0.0 mV    / STD =   13.7 uV  
Channel F5    : Offset =   -0.5 mV    / STD =   13.8 uV  
Channel Cz    : Offset =    0.2 mV    / STD =   13.9 uV

END OF $batch$ DEMO

Thanks! I hope it was useful!

Enjoy Batch.

Main author: R Sanchez-Todo (roser.sanchez@neuroelectrics.com)

Contributors: are welcomed